Running non-native executables with qemu
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use qemu-(arch) to run init and telinit built for (arch):

    qemu-arm ./init

Guest-arch syscalls will be passed to the host kernel, with necessary
ABI conversion performed by qemu. To check syscalls being passed, use

    qemu-arm -strace ./init

This simple approach works best for quick viability checks, tests
(those from test/ directory) and simple debugging using warn() calls.


Debugging non-native executables with qemu and gdb
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The trick is to use qemu built-in gdb server.
Start executable being debugged first:

    qemu-mipsel -g 1234 ./init

then connect gdb for debugging:

    mipsel-linux-gnu-gdb -ex "target remote tcp::1234" ./init

Note gdb must be target-gdb! Native one is not likely to be useful.

TCP port (1234 in this example) is arbitrary. Check gdbserver(1)
for more info.

Beware of qemu(1), the page I have describes qemu-system-(arch) only,
*not* qemu-(arch) which has different options. Try running qemu-(arch) -h
instead.


Running init in a complete emulated system
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Some aspects of init can only be tested on a dedicated linux system
with the init running as pid 1.

The easy way to do this is to use buildroot to make a bootable system
image with sninit in it and then run it in qemu-system.

Get buildroot, and make a minimalistic system for your target architecture.
Using configs for some popular board is a good idea. Make sure busybox
is enabled (it should be). It's ok to disable any other packages at this
point. Build the system, and make sure it works in qemu.

Example command (buildroot, arm, versatilepb board):

    qemu-system-arm -machine versatilepb \
        -kernel output/images/zImage \
        -hda output/images/rootfs.ext2 \
        -append "root=/dev/sda"

Next build sninit for the target machine. This can be done
	(a) within buildroot
	(b) outside of buildroot using buildroot toolchain
	(c) outside of buildroot using unrelated toolchain

If you do (b) or (c), copy executables and inittab
to $BUILDROOT/output/target/ and re-run make to get them
into rootfs.img.

Regardless of the path chosen, make sure you have sninit
executables in your rootfs image:
	$BUILDROOT/output/target/sbin/init
	$BUILDROOT/output/target/sbin/telinit
	$BUILDROOT/output/target/etc/inittab

When building sninit for complete system test, make sure
to disable DEVMODE, otherwise it won't be able to reboot
the system.


(a) Building sninit as a buildroot package
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Check files in misc/buildroot/, in particular README there.
With some luck, you should be able to build a complete,
qemu-bootable, sninit-controlled linux system image with
a couple of commands.


(b) Building sninit using buildroot toolchain
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Set CC and AR in config.mk, or supply them to configure:
	CC=BUILDROOT/output/host/usr/bin/TARGET-gcc
	AR=BUILDROOT/output/host/usr/bin/TARGET-ar
Substitute BUILDROOT and TARGET to match your particular
configuration.

Make sure ARCH= in config.mk matches TARGET,
or leave it empty to use buildroot's libc.

That's all actually, run make and it should work right away.


(c) Building sninit using unrelated toolchain
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This trick is only recommended for bundled libc builds.

If you have *any* toolchain for TARGET installed in the system,
you can use that to build sninit executables to use them
in buildroot system. Set CC, AR and ARCH in config.mk, and build
as usual.


Debugging init in a complete emulated system
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You will need a kernel and bootable rootfs image containing
busybox, gdbserver and sninit. Refer to buildroot docs for instructions.

Build a target gdb for your host system too (BR2_PACKAGE_HOST_GDB=y)

Enable serial port, and change kernel command line
to start sh instead of sninit (init=/bin/sh):

    qemu-system-arm -machine versatilepb \
        -kernel output/images/zImage \
        -hda output/images/rootfs.ext2 \
        -append "root=/dev/sda init=/bin/sh" \
	-serial pty

With -serial pty, qemu will report allocated PTY slave device, typically
something like /dev/pts/N. Boot the target system and run

    gdbserver /dev/ttyAMA0 /sbin/init

in sh prompt. Use the right TTY name: most systems provide ttyS0 or ttyS1
but for ARMs it's ttyAMA0.

Start gdb on the host system:

    cd BUILDROOT/output/build/sninit
    BUILDROOT/output/host/TARGET-gdb -ex "target remote /dev/pts/N" ./init

Adjust sninit build directory and use the right pts device, the one that qemu
reported before. If everything goes well, gdb will show the process stopped
right after _start: symbol.
Set breakpoints if necessary and use "cont" to unfreeze the process.

Refer to gdbserver(1) and remote debugging sections in gdb manual
for more info on using "target remote".

Serial link is typically the easiest one to set up in qemu, but in some cases
tcp may be a viable option. See qemu(1), especially on -net and -serial.


Debugging over a tcp connection
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This may be easier to use than a PTY, but requires several things (qemu,
guest kernel and guest userspace network setup) to work properly.

Set up emulated network device and forward some port, say 1234, from host
to guest system. Check by running

	nc -v -l -p 1234	# guest side
	nc -v localhost 1234	# host side

The connection should work. If it does, use :1234 instead of serial device
in gdb commands:

	# guest side
	gdbserver :1234 /sbin/init
	# host side
	.../TARGET-gdb -ex "target remote :1234" ./init

Check

	misc/buildroot/board/sninit/qemu_aarch
	misc/buildroot/board/sninit/root.aarch/etc/setup

for sample configuration.


Debugging pid 1 specific features
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Running init under gdbserver means it won't get pid 1.
To examine reboot, fool it by setting S_PID1:

    (gdb) break main
    (gdb) cont
    Breakpoint ...
    (gdb) set state |= 1

There is no easy way to emulate orphans at this moment.
If really necessary, try adding prctl(PR_SET_CHILD_SUBREAPER) to setup()
and spawning some children that use setsid(). However, if things got this far,
init's child-reaping functions are not likely to be buggy.


Reboot and wakeup
~~~~~~~~~~~~~~~~~
Switch to qemu monitor (Ctrl-Alt-2 in the graphic window)
and use these commands:

    sendkey ctrl-alt-delete
    system_wakeup

Ctrl-Alt-Del naturally only works if particular VM has keyboard; the MIPS
one doesn't. The event itself is handled by kernel, which is turn sends
SIGTERM to init.

For more info on this, use "help" command in the monitor, or check qemu docs.
The scroll the monitor, use Ctrl-PageUp/Ctrl-PageDown.


Caveats
~~~~~~~
Qemu in linux-user mode (qemu-arm, as opposed to qemu-system-arm) does not
check nor translate hidden syscall arguments like sizeof(sigset_t) in
ppoll, and always supplies the right values to the host kernel.
This behavior may mask bugs in those arguments.

Typical presentation: executable works well on qemu-arm but fails
with EINVAL from ppoll on qemu-system-arm.

When working with syscalls, do not forget to test on qemu-system-(arch)
even if everything works fine in qemu-(arch).
